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I <1.  HIT  VOMOt  fC« 
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Nonlinear  systems:  system  identification;  remote  sensing:  estimation 
theory;  optimization;  approximation  theory;  Fock  spaces;  Volterra 
functionals:  operator  theory;  functional  analysis. 

I*.  *911  n*CT  iCrnttmt  tm  mvwm  MM  W MccMar  *4 IMMII*  »r  Mm* 


,-sc  f<f 


A method  is  presented  for  the  identification  of  a nonlinear  syston  repre- 
sented oy  an  operator  V:E->Y,  where  the  input  space  E is  a separable  Hilbert 
space  over  the  field  of  complex  numbers  and  the  output  space  Y Is  the  Sobolev 
spaeev^p^of  comp  I ex -valued  functions  y on  an  interval  X of  the  real  line  , 
such  that  <^y/di'',  k»0 n-l,  are  absolutely  continuous  and  j"^/dli'^L^(l).'’4 


DO  I'SITn  1^3  cofTioM  or  I MOV  It  It  OMOvsTt 

i/N  aio}«kr.oi4.*Mi 


f CulHt^  CLAttliyATlOA  0— 


IteUMrr  ekASMn«*TION  of  fmt  ^«aK  pnm  Oma  utmao 


Our  identification  scheme  is  based  on  a given  set  of  input»output  pairs 

IKUj.yps  «jr€E,y.€ff  i^l,...ni)  and  on  an  appropriate  description  of  the  class 
: to  which  V is  assumed  to  belong,  a description  which  permits  us  to  incorporate 
In  the  formulation  of  C our  a-prlori  knowledge  of  the  properties  of  V. 


20.  ABSTRACT  (CONTD.) 


Let  y(t) 
to  an  Input  u. 


(Vu)(t)  denote  the  output  at  time  t corresponding 
We  express  this  In  the  form  y<t)  > V (u).  where  we  assume  that 
V belongs  to  the  Fock  space  ^(E)  of  order  P over  E.  If  E • 1,(1),  V can 
be  represented  by  a Volterra  functional  expansion.  Ve  show  and  use  the^ 
property  that  F.(E)  Is  a reproducing  kernel  Hilbert  space.  Also,  we  construct 
a Hilbert  space'^B^  ' 
characterize  the 
in  B2(I,Fp(E)). 


' (r,Fp(E))  of  nonlinear  operators  from  I to  F.(E)  and 
operator  class  C to  which  V belongs  as  an  ellipsoidal  class 


The  above  developments  permit  us  to  obtain  the  solution  to  our  nonlinear 
system  Idontif icat Ion  prpbj^  as  the  solution  to  an  appropriate  minimum  norm 
probIeml{x,^Ml,Fp(^)i'“  Procedures  for  obtaining  both  the  noncausal  and  causal 
soluttSns  ana  giv%n.  We  elso  introduce  the  concept  of  "c-causa 1 1 ty" , which  Is 
weaker  than  that  of  causality,  and  derive  an  x-causal  solution  to  our  problem. 
The  case  when  measurement  errors  are  present  is  finally  considered. 

The  results  ?re  illustrated  by  the  computer  simulation  of  a simple  example 

In  which  very  good  agreement  with  the  theory  is  obtained  over  e wide  tIme-Inter 
val . 


J ••  j ::-  moi 


itcumTT  a*  Tuts  f ks(r«M«  o«« 
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1 . Introduct ion 

•k 

Let  S denote  a system  represented  by  a nonlinear  operator  V:E  - Y, 
where  E and  Y are  appropriate  Hilbert  spaces  called  respectively  the  "input" 
and  "output"  spaces.  From  now  on,  the  norm  and  inner  product  in  a given 
space,  say  H,  will  be  denoted  by  and  jj  . ||^,  the  subscript  being 

omitted  when  the  space  referred  to  is  clear  from  the  context. 


In  the  present  paper,  we  consider  the  problem  of  identifying  S and  hence 
V,  based  on  a given  set  of  input-output  pairs  ((u.,  y.)]u.  € E.  yj  € Y, 

i = 1 m } (called  "probing  input-output  pairs")  and  on  the  class  C to 

which  V is  assumed  to  belong. 

Our  framework  is  such  that  the  operator  class  C is  defined  by  assuming 
a "finite  gain"  property  for  the  operator  V and  taking  into  account  the 
smoothness  properties  of  the  output.  Furthermore,  members  of  C are  not 
required  to  be  known  up  to  a finite  set  of  parameters.  For  this  reason,  we 
regard  the  system  identification  approach  developed  here  as  being  a "non- 
parametric  approach." 

Specifically,  we  formulate  the  nonlinear  system  identification 
problem  in  its  fullest  generality  as  follows: 

For  a given  "7  € C,  let  denote  a norm  on  the  operator  class  C 

which  measures  an  appropriate  error  in  the  approximation  of  V by  members  of 

A 

the  entire  class  C.  Then,  under  no  measurement  noise  conditions,  V is  our 

best  estimate  of  the  operator  V (which  is  to  be  identified)  if  it  is  the 

r m I -Acoession  For ^ 


solution  of  the  following  problem: 


DD'C  TAB 

Unsiinciu:-iccd  LJ 

Ju;.t.ific  tior 


* By  "nonlinear"  we  mean  "not  necessarily  linear." 


Pi-  • V 


Problem  I.  Given  u.  gE,  y.  i“l,  ...,m  and  an  appropriate 

operator  class  C,  find  V as  the  solution  of 


min{e^(V)} 
V ^ C 


'\/(u.)  “ Y; . i “ i , , m . 11 

If  the  output  measurements  are  corrupted  by  noise,  we  model  the  rela- 
tion between  the  probing  input-output  pairs  (u.,  y.),  i = 1,  m,  by 

y.  - V(u.)  + V. , i = I (1.2) 

irir 

where  v j Y satisfy 

(vj,  Vj)^  »q.6|j.  i.  j = 1 m,  (1.3) 

q.  being  positive  constants,  and  6.j  =•  Kronecker  delta.  The  set  of  equa- 
tionsfl.3)  may  be  interpreted  as  there  being  no  correlation  between  the  noise 
present  in  the  ith  and  jth  measurements.  In  this  case  our  best  estimate 

A 

V of  V is  the  solution  of: 

Problem  2.  Minimize 

J(V,_i:^  q"'  llV(u.)  - y.ljj).  (1.^) 

where  J is  a criterion  which  optimizes  the  estimate  with  respect  to  the 

class  properties  of  V as  well  as  noise.  |] 

In  what  follows,  we  assume  that  the  input  space  E is  a separable  Mil- 

2 

bert  space  over  the  field  of  complex  numbers;  Y is  the  Sobolev  space  H (I) 

n 

of  comp  lex- valued  functions  g on  an  interval  I of  the  real  1 ine  such  that 


* Ends  of  formal  statements  will  be  signified  by  the  symbol  ||  . 

**  If  the  noise  belongs  to  some  bigger  space  than  Y,  then  we  define  Vj  as 
the  projection  of  the  noise  on  Y, 
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(k)  A k k 

g ( “ d g/dt  ),  k » 0,  I 0-1,  are  absolutely  continuous  and 

g^”^  ^ L^(l);  and  V belongs  to  a Hilbert  space  of  nonlinear  operators 
2 

B (1;  F (E)),  which  we  call  a Bochner-Sobol ev  space  of  order  n over 
n p 

F (E) , where  F (E)  is  a Fock  space  of  order  p over  E, 

P P 

2 

In  sections  2 and  3,  we  define  the  spaces  F (E)  and  B (I.F  (E)), 

P n p 

and  present  mathematical  developments  pertaining  to  them  needed  in  later 

sections.  We  show  that  F (E)  is  a reproducing  kernel  Hilbert  space  and 

we  establish  useful  links  between  F (E)  and  B^(I,F  (E))  on  one  hand,  and 

P " P 

their  particular  manifestations  in  the  form  of  spaces  of  Volterra  func- 
tionals and  of  Volterra  operators  on  the  other  hand. 

Section  4 is  devoted  to  a detailed  formulation  and  solution  of  Prob- 
lem I for  the  general  case  of  a noncausal  operator.  In  Section  5,  these  results  are 
particularized  to  the  causal  case  snd  (after  the  introduction  of  the  concept  of 

"€-  causality")  to  the  <-causal  case.  Problem  2 is  solved  in  Section  6 and  an 
example  presented  in  Section  7. 

The  present  approach  to  the  system  identification  problem  is  similar 
to  the  one  proposed  by  de  Figueiredo  and  Caprihan  [1],  [2]  for  the  identi- 
fication of  linear  systems,  with  the  basic  difference  that  in  the  linear 
case,  the  space,  to  which  the  operator  to  be  identified  belonged,  was 
assumed  to  be  the  space  of  "trace  class"  operators. 

Referring  to  contributions  of  other  authors  relevant  to  the  develop- 
ments in  the  present  paper,  special  mention  should  be  made  of  the  early 
work  of  A.V.  Balakrishnan  [3]  and  others  (see  references  in  [2])  on  the 
identification  of  nonlinear  systems  from  input-output  data,  as  well  as  the 
recent  contributions  of  F.J,  Beutler  and  W.L,  Root  (see  [4]  and  references 
therein)  on  the  Identification  of  linear  as  well  as  polynomic  systems  much 
along  the  lines  of  the  present  paper.  Other  papers  on  related  topics  are 


the  ones  by  J.L,  Franklin  [5],  E,  Mosca  [6],  and  W.  Porter  [7]  [8]  [9], 

We  now  proceed  to  construct  the  mathematical  framework  used  in  the 
solution  of  Problems  I and  2. 
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2.  Deve lop-nents  Pertaining  to  Fock  Spaces  and  Vol terra  Expansions 

Let  E be  the  separable  Hilbert  space  introduced  previously,  and 
denote  by  B^(E;^)  (where  (|  denotes  the  space  of  complex  numbers)  the 
Banach  space  of  n-l inear  symmetric  bounded  forms  A ; ExEx...xE  ^ 


|A(U|.---u^)|sM||  u,||  ...||u^| 


for  some  M € R,  Kor  any  A 6 B^(E;(l), 


define  a new  functional 


P.  : E - C by 
A 


;iiu!  n norm  on  P.  by 
A 


p (u)  = A(u,.  ..;.0 


The  space  of  functionals  L 1J^(E;C  )3  is  a nermed  litiear  sp,.cc',  and 

We  denote  by  P(*^E)  the  Banach  space  of  coni  inuous  n- hijnei;cnoous  poly- 
lu'iiiial,'.  I'bt.iiiu'd  by  comp  le  I i nj’  b^^(i'.;C)j  it:  i he  norm. 

For  A B^(E;t)  deline  the  norms  ll'iljj  '»'>d  il-'ijj  •"'s  ImIIow.s  (the  seb- 

.scripts  H and  B .stand  for  Hilbert  and  H.uiich  respectively  and  do  not 


relcr  to  specific  sp.ices) 


1-^lhi 11mJ=i 


iMUj . 


sup  |A(g C4)|  , 

!u!l=i 


and  it  is  immediately  obvious  that 


Ifi  - 


It  is  not  difficult  to  conclude  that  for  symmetric  n-linear  forms 

one  can  bound  1|a|1j^  s c^  some  constant  c^  dependent  only  on  n 

[10  J . So  ll'lL  and  j]  • IL  are  equivalent  norms  on  the  space  B (E;(f). 

H 0 n 

I.ei  ll*'  denote  the  tensor  product  of  a Hilbert  space  H with  itself 

n times  [11,1-]  and  let  Cj^Q  e^Q.  . .®  e^  be  a decomposable  element  of 

e"  where  e^  € E.  Then  froon  tog  definition  of  e"  \X  [ollows 

lle^Sej...  ® ejl^n  - He^H  . . .He^H  and  (e^Q  e^Q. . .e^)  [m^,...,'J^]  = 

<‘=l'^>E---^®n*'Jn^E  ‘'l'---''"‘n  ^ [•.•••.•]  is 

a member  of  the  n-th  order  Cartesian  product  of  E.  Inl-rccli^cS  i; 

^1®  (‘^l*‘>E---(‘'n’‘^E-  i maps  the  decomposable  ele- 

ments of  E*^  into  P("e).  It  is  clear  that 

||i(e^®.  .e^)||  - l!(e^.-)E...(e^.-)  Il^  ^ lle^ll  • • • l|e  J . (2.6) 

Dcfinfc  i by  linearity  on  all  finite  linear  combinations  of  decom- 
posable elements  of  e"  and,  using  equation  (2.6), it  is  easy  to  demon- 
strate [12,13]  that  i is  a bounded  linear  map  of  finite  linear  combina- 
tions of  decomposable  elements  oi  E*'  into  P("e) . Finally,  by  con- 
tinuity extend  i to  all  of  E*^.  Define  a new  map  S : E*^  -•  E*^  by 

n 


I 


(2.7  ) 


where  P^  is  the  permutation  group  of  n integers  and  C acts  on  decom- 


posable elements  of  E by 


0(c,Qe-...©e)  = c ®e  ®...Qe 
12  n a,  0_  O 

12  n 


(2.  S) 
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E is  separablCj  tl-,iire  ib 

orthonornijl  basis  for  E.  Tlicn  it  is  well  known  [t^]  that 

{cp.  © cp , <S.  . .®  ;p.  j ’ * ’ is  an  orthonorinal  basis  for  E , Clearly, 

S ^2  ij-l,...,ij^-l 

CO  ® 

0 may  be  lie  fined  on  all  of  e"  by  C ( / •••  / ■ ■ T Q.  . .Qp. 

\ L.  L.  i 

1=1  n“l 

go  CD 

)....)  c,  cp.  ®...cp,  , (2.  &)pTnd  iurthermore , one  can 

. ^l>--*\i  ^0,  *0 

1 ,•!  i «1  1 n 

1 n 

show  that  C is  a bounded  linear  operator  on  k"  into  e".  It  then  follows 
[IZ.]  that  defined  in  equation  (2.7  ) is  a projection  operator  in  e”, 

+ 

ie.  « S^.ind  (where  the  superscript  t denotes  the  adjoint). 

The  subspace  fs  called  the  syiainctric  tensor  product  of 

E of  order  n.  For  the  case  of  E = L^(r')^e'’  is  just  L^iR’)®.  ..®  L.,(ii)  and 

it  can  be  sl'.own  that  there  exists  an  isomorphism  J hetween  e"  and  L.,(r'^) 


suili  lhaC  J sends  the  clement  \ 


e e Q.  . .Se . into 

h "n  h 


'l’--''n 

h(L,,...,t  ),  wherc^  h' t L-(r'')  and  {e.l.  , is  an  orthonormal  basis  for 
1 n 2 1 1=1 

I.  {K^.  S e”  i.s  then  just  the  subspacc  of  L.,(R")  of  function.s  left  invari- 
2 n . ‘ i. 

.lilt  under  any  permutation  of  their  variables.  Furthermore,  one  can  show 
[IZ]  chat: 


Z "i,.....i 

, .1  n 1 n 

^n 


r’  r’ 


*•  ..  » I r 

» 01  CC'oC'.es  ' >■  cc  _ •'  ' 
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where 


-[('-*)  “ J e*(t)  a(L)  dt, 


(2. 1C) 


where  € L2  (R)  • To  complete  our  construction  we  look  at 


Che  space  i(S^E")  C P(*^E)  . Certainly  i(S^^E^)  is  a linear  space  and 


since  i is  1-1  on  S E^  ’ [13]  wo  Cwn  define  an  in.ner  product  between 


two  elements  u,vC  ^ 


= (i  ^ i \v))^n 


(2.11) 


Now  we  invoke  tlie  followinp,  theorem: 


Tlieorem  2 . 1 


If  i is  a 1-1  bounded  linear  operator  frem  a Hilbert  space  H into 


a Baiflich  space  B then  the  image  i(H)  C B is  a Hilbert  space  with  the 


inner  product 


<u,V>  = (i■^u).i■^V)), 


(2.12) 


Proof:  It  is  clear  that  i(H)  is  a vector  space  in  B and  furthermore 


it  is  an  inner  product  space  with  inner  product  (2 . 12j.  To  show  that  i(H) 
is  actually  a Hilbert  space  we  must  demonstrate  co.mpletoness . Let  [h^} 


be  a Cauchy  sequence  in  i(H),  ie  V e > 0 3 N(c)  such  that  (b^-b^ ,b^-bj)<  C 


for  i,j  i N . But 


■<b^.bj,b,.b.>  . (r'(b,-b.),i-‘(b.-b,))„  - 


(i‘\bp  - i‘\bj),  i'\bp  - i‘^b.))u 


(2.1?) 


so  that  [i  \b^)}  is  a Cauchy  sequence  in  H.  But  11  is  complete  so 


there  exists  h € H such  that  i (b^^)  h.  Therefore 


9 


I 


<b.-i(h),b.-i(h)>  - (i*^b.-i(h)),i'\b.-i(h)))jj  » 

(i'^bp  - h,  i'^bp  - h)  - 0 . II  (2.10) 

We  can  now  say  chat  iCS^E*^)  is  a Hilbert  space  of  n-horaot;eneous 
synsiiecric  polynomials  on  E.  We  denote  i(SE")  by  Pjj('’e)  and  is 

known  in  the  literature  as  the  Hilbert -Schmidt  polynomials  on  E [li]. 

From  now  on  we  will  denote  the  inner  product  in  P (^E)  by  (•,•),.  • We. 

H L 

n 

state  the  following  propositions  connecting  tensor  products  of  spaces 
and  the  Hi Iber t -Schjnidt  polynomials. 

Pronositicn  2.  I 

Let  M be  a measure  space  and  E = L^iM),  Chen  P € Pjj('^E)  if  and  only 

if  there  exists  h € L-(Mx...>M)  such  that 
n 2 

P(M)  . J J , (2.1.'-) 

Mx.  . .xM 


h is  syncr.etric  and 
n . ^ 


(MX,...,XM)  • 


(2-10 


Proof : Sec  Reed  and  Simon  [IZ]  . 


Proposition  2.2 

Given  an  orthonormal  basis  e-’ch  P € Pjj('^E)  is  uniquely 

expressed  as  a limit  in  11 norm  by 


= ) c.  p....e. 

, L,  1,  1 

i t 1 ’ n 1 n 

1 II 


(2.r) 


with  syiimietric  coefficients  c^  . € C and 


I 

j 


I 


I 


We  - I ,lv ,1  ■ 

n i,  .....  1 1 n 

1 n 

Here,  e'  (^)  - (c^.'.^)p  for  E.  If  R € and 


y ‘i,  c'  .....c! 


(2.!5) 


i^....in 


■,r)e  “ y i ‘‘i  i 

^n  , ^ ^1 




Proof : Sec  Dwyer  [j^J  • 


(2.2o; 


We  will  now  define  the  Fock  Space  [12.1?)  i ^ ] of  functionals  which 
will  be  the  mathematical  framework  within  which  Volterra  expansions  will 
be  cons ider ed . 

Definition  2.1  Let  p be  some  fixed  real  number  >0.  The  Fock  space 
of  order  p,  denoted  F^ (E) , is  the  space  of  sequences  ^ f ^ . f ^ . f 2 . . . . ) where 
f 6 d and  f € P,,('^E),  such  that 

"^n"E  ^ 

n=0  "•  ’’  “^n 

It  is  well  known  []2]  from  the  construction  of  Cartesian  products  of 


Hilbert  sp.ices  tliat  F^  is  a Hilbert  space.  We  will  denote  the  inner 
product  between  two  elements  f = (f^.f^,...)  and  g = (SqjSji-'-)  of  i 


00  ^ 

<f.g>F  ° y ^ ^^n’^’n^ 


(2.22) 


What  we  now  will  show  is  that  F (E)  can  be  emsidered  to  be  a space  of 

★ 

functionals  on  E by  the  relation 


f(M)  . I 


? f„(M) 


(2.23) 


Thus  an  element  ft^(E)  will  be  viewed  both  as  a sequence  such  as  in  the 
Definition  2.1  and  a functional  on  E the  evaluation  of  which  is  defined 
by  (2.23). 


M c I 


II 


2 

for  f € Fp(E).  Suppose  that  we  have  an  clement  f^  € Pjj(  E)  ; then 
we  know  by  Proposition  2.2  that 


£ - y *=1  i “l 


(2.24) 


lj,i2 


and 


2''E„ 


I 


(2.55) 


But  is  also  an  element  of  P(  E)  .so  wc  can  consider  the  Banach  norm 
(equation  2.50  of  £ . By  definition,  Hf^Hj^  - sup  |f2(u)|  and  by  (2.2i), 


.11,-1 


2"B 


Slip 


I \ c i c!  (u)  e[ 

1 ' ^1’  2 M 


' f It 


; («)  1 • 

2 


(2.1^) 


Sinco  fe  ]“  , is  a l)usls  tor  E we  can  write  U - c^  ,o 
t i*l  ^ ^ ^ 

i^l 


(2.17) 


where 


“ ('-i,o^)g  ■ o'Cj)  . 

Applyini;  the  C.iuchy-Schwar tz  inequality  to  the  visht-hand  side  of  (2. It), 
we  i’.e  t ; 

) c c'  (u)  e'  (u)|  J I ‘‘i  'i  I ■' 

S h >h  h ^2  , 1 2 'l  2 


I j > I2 


i,.!. 


^1*4 


4’4 
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u , 


p,„of.  Accord  in.,  to  the  precedi  nq  lenm,a,  e . 


1-1 


m,  are  1 inearly 


12 


).  I 'i  I I 

^ I » 2 


^ 1 1 » i 


1/2 


/ 


ZK, 


2 1/:: 


/ 


1/2 


) k,.  I‘  . (2.?S) 


U“1 


u, 

i“l 


aiul  since  \\u\\  - 1 in  (2.26),  this  implies  ) Ini.l'  - 1.  Using  this  lest 

equation  in  (2.2.*5)  we  finally  I'btain 
sup  I y c *.•[  (u)  e'  (u)  I S ( y I C 

Li  “1  , n ^2  h S h ‘2 


(2 . 1'^) 


a 1 


2"K,  • 


^1*^2 


ij . t., 


Steting  these  r<‘sult.s  in  n preposition  for  arhitr.iry  n,  we  pet 
I’r  012.0  s ijji  on  .1 

U e 1'„(’'k)  then  Ilf  Jlj,  S 111  II,, 


(2.  to) 


where 


f II,  •=  sup  |f  (u)|  . 
I'  » II. .11  ' ir  • ' 


,hll,-i 


We  .ire  now  in  po.sition  to  sluv  that  rel.ilii'ii  (2.23)  defines  .i  iKHiiuleJ 
tvincUonal  on  K.  It  f r^i^K) , then 

|f,/'-l)|  » IMl"  |f^  ) I g ||ei|l"  Sup  h\^(v)| 

II  V«rl 


- ihir  1UJI3  «-■  HI"  Ilf 


n"l-  • 
n 


(2.31; 


We  can  sec  from  (2.23),  (2.30)  and  (2.33)  that 


j 

a 


OP 

V 

f (u) 
n 

‘P  f (U ) 

1 <-  \ 1 - 

1 -N 

'p" 

n-O 

L 

n^  o 

n.' 

1 ^ Z 1 n; 

n-0 

1 V ti 

n ; P 

.11^11'" 

y«/ 

\ p"' 

1/2 

-’p 

lifjr 

p'’n.' 

/ I 

Z*  nT  " 
no  j 

z 

' n-e 

p"  "‘.Jif:  ■' 


(2.32.) 

'n  ii)  l,/2 
- P 


1 


1 


I 


1 3 


But  since  we  are  dealins  with  f (.  FpCf-).  we  can  write  (2.320  as 

2 

|£(u)|  s e • llfllp.  < Oo  • (2.33) 

P 

This  allows  us  to  state  the  major  theorem  of  this  section. 

Theorem  2.2 

If  f € F (E)  then  f • ^ ■ n Is  an  entire  function  of  bounded  type.  ! 

n“o 

ie.  f takes  bounded  sots  into  bounded  sets.  If  l)''f(u)  denotes  the  n-th 
Frechet  derivative  of  f at  U then 

u"f(0)  “ . (2.34)  I 

The  class  of  functions  F (E)  is  a Hilbert  space  with  the  inner  product 

P 

CO 

<•,■)  Riven  by  (f.R)  - y p"  ^ <l)"f(0).D"g(0))g  . H (2.3?) 

P P ^ 

e n=o 

Proof;  Equation  (2.3-4)  establishes  the  first  sratement  and  the  proof 
of  the  remainder  of  the  theorem  may  be  found  in  Dwyer  [i3].  1, 

r 

The  construction  of  the  space  Fp(E)  for  an  arbitrary  Hilbert  space 
E is  due  to  Dwyer  [13.14- 

The  construction  of  F (E)  when  E “ L„(M),  where  M is  a measure  space  | 

P 2 [ 

was  well  known  [12j  before  Dwyer.  The  characterization  of  Fp(E)  when  | 

E « 1.2(R)  fs  still  a most  useful  .ind  easily  understood  Fock  space.  ^ 

Before  we  conclude  this  section,  lot  us  introduce  a map  which  will  | 

later  prove  to  be  exceedingly  useful.  If  u C E then  the  n-th  order  poly-  ! 

(n)  _ t 

noinial  (•)•■• '-1^  (’ ) E is  lortainly  .m  eleaient  oi  I’u(  E).  Define  j 

exp(u)  € F^(K)  to  bo  tlie  fund  ional  i 

I 


27 
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f = I 


where  f = ( • ) • • -vi  ( ■ ) 

n 


It  is  clear  that  for  / € E 


(exp(u))(v)  = e 


and  that 


llexp(u)|l  - e 

Now  if  f € P„(''e),  then  we  may  write 
n n 


y c.  ef  . . .ef 

Lt  » • • • » 1,  1 


1, 1 . . .1 

1 n 


n 1 


Using  equation  (2.27)  we  get  for  V 4 € E,  u'(  • ) • • • . ) = 


i^....in 


1 n "1 


/ / 
i 


Therefore  by  Proposition  2.2  we  conclude  that 


(n) 

(f  .u'(.)...u'(.)). 


n . .1  n 1 n 

t^,...in 


We  can  now  state  the  following  important  result. 
Proposition  2.4 

If  f € F (E)  and  L € E,  then  f(s)  = <f,  exp(-^lV  . 

P p Fp 

Proof : By  definition  and  equation  (2.41), 


(2.36) 


(2.37) 


(2.38) 


(2.39) 


(2.40) 


(2.41) 


(2.42) 


28 
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«“'-I  ■ i — = 


n*o 


(2.4ta) 


It  is  worthwhile  to  reilerive  cqujtion  (2.3i;  in  light  of  the  above  pro- 
position. It  is  clear  that 


I f (U)|  - < f ,cxp(yi>j,  I 


p'-i'F 


(2.43) 


P P • p 

by  (2.^2.)  and  the  Cauchy-Schwar t z inequality.  By  equation  (2.35)  we  may 
rewrite  (2.4^)  as 


|f(^)|  ^ 11  flip  el 


lull^/2p 


(2.4^) 


which  is  just  inequality  (2.32.).  We  have  actually  shown  more  in  Proposi- 
tion 2.4  than  we  set  out  to  do.  We  have  proven  that  point  evaluation  is  a 
continuous  functional  on  the  sp»cc  Fp(E).  Equation  (2.42a)  identifies  the 
representer  of  the  point  evaluation  at  u as  the  element  exp(^).  For 
future  reference  we  state  this  observation  in  the  form  of  a theorem. 

Theorem  2.3 

The  Fock  space  Fp  (E)  is  a Reproducing  Kernel  Hilbert  space  witli  tlie 

reproducing  kernel  e for  M,v  6 E.  || 

Suppose  that  our  Hilbert  space  E = L2(R).  What  would  the  Kock  .Sp.ice 
of  L2  (R)  look  like?  Proposition  2.1  tolls  us  that  f^  € Pjj('^L2(R))  has 
I he  form 


29 


I 


(2.4 


V 

Tl\us  since  £ ('J)  “ ^ >'  '^®  •’’^y 

n“0 


£(^)-h^  + j|  jhi(ti)u(t^)dt^  + ^ IK(ti.C2)u(ti>u(tpaL^dt,  + 

' R . j,2 

3T 


R3 


where  h^6t.  hi  € ^(K).  h^  t L^CR  ).  etc. 


By  definition, 


I ?fT  J „n'J  l^"r 


.,t„)|2  dt,...dt^  . (2.- 


n»o 


A question  which  might  naturally  arise  is  how  much  error  we  incur 


£„(u) 


by  truncating  a Vol terra  series.  In  other  words,  if  f (w)  = ^ 7 


n=o 


1-(U) 


and  we  approximate  ^ ^y  2^  "^"i — how  tan  we  hound  tin;  error  of  such 


■ in  .ippr ox i mat  i on  in  a poinewise  fionse?  We  know  that 


N f (M)  - f„(u)  r ,,  fj  lUJl 

I'Od)-  2.  I 'll  “ I " ). 

n«o  n»N+l  n”N+l 


2n 

V ll^ii 

1/2 

V ^ 

2 \ 

111  11 

=N>1  P 

Z,  n! 

, n“N+l 

•1  n"  E 

2n  1/2 


0 


N+1 


p n; 


1/2 


17 


So  l£  uc  Je{i\ic  the  truncated  exponential  function 

■ I 


k 

lU. 


k; 


7 7 


( 2 ^'i) 


k»N 


wi'  (..111  sjy  that  Cho  error  satisfies 


M f fy) 

“'*'-1  -f-- i '■ 


n.o  P 

The  goodness  cf  the  truncated  approxiinac i on  is  preserved  only  in  a ..nail 


lie  ighhorliood  about  the  origin  and  deteriorates  for  laige  values  of 

lluil,. 


P 


I 
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3.  Hilbert  Spaces  of  Nonlinear  Operators 

We  now  proceed  to  introduce  spaces  of  Hilbert-valued  functions  on  an 
interval  I of  the  real  line. 


Def in i t ion  3, I B (X.  F (E))  is  the  space  of  functions  (operators)  V 
o p 

from  I to  Ep(E),  where  V is  strongly  measurable  with  respect  to  ordinary 
Lebesgue  measure  u on  1 and  satisfies 


; ii''(t)ir 


<ii  < 


F (E) 

P 


(3.1) 


We  will  often  denote  the  value  V(t)  € F (E)  by  V , and  also  replace 

P 

by  dt  with  the  understanding  that  all  integrations  are  in  the  senses  of 
Lebesgue  or  Bochner. 

Note  that  if  u € E and  y is  the  function  on  I defined  by 


y(t)  - V^(u), 

then,  since  'according  to 

^(y(t)|‘  dt  - ,^|V^(u)l^  dt  s 


(3.2) 


exp(||u|rVp) 


it  follows  that  y € L (1). 


F (E) 

P 


dt, 


(3.3) 


The  following  is  easily  established. 


Theorem  3.1  [15  | . B (T,F  (E))  is  a Hilbert  space  unaer  the  inner  product 

o p 


(V.W)  -/(V  .Wl  dt 

J ‘ ‘ F (E) 


.11 


(3.4) 


In  connection  with  some  applications,  it  is  appropriate  to  introduce 


2 

smoother  versions  of  the  space  B (1,F  (E)). 

o P 


32 


19 


A function  g:I-  F (£)  is  said  to  be  strongiy  differentiable  at  t F I 
P 

if  there  Is  an  element  Dg(t)“g'(t)  € F (E)  such  that 

P 


Lim  II  gitihj^Ct)  . 
h-O 


(3.5) 


Higher  order  strong  derivatives  0 *g(t)-g^ * \t) , i > 1,  are  similarly  defined. 


Definition  3.2.  B (I.F  (E))  is  the  soace  of  functions  U from  T tn 
n p 

F^(E)  such  that  V,  V',...,\/^”  are  absolutely  continuous  and  belong  to 
B^d.F  (E)),  and  E B^(r,F  (E)).|| 

Op  Op" 


As  in  the  case  of  Theorem  3.1,  we  have: 

2 

Theorem  3.2.  B^(r,F(E))  is  a Hilbert  space  with  the  inner  product 

(V,W)  - r a,  / dt.  (3.6) 

b2  i-0  ' I F (E) 

®n  P 

where  a.  are  positive  constants. || 
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Nonparametric  Nonlinear  System  Identification  in  thg  Noiseleaa  Casa: 
Non-Causal  Solution 

We  now  return  to  the  nonlinear  system  Ident i f icat ion  problem,  and 
assume  that,  in  the  input-output  relation 


y - V(u) 


(4.  I) 


satisfied  by  the  system  S to  be  identified,  u and  y belong  respectively 
2 2 

to  E and  H (I),  and  V c B (I,F  (E)).  For  simplicity  we  will  denote  the 
n n p 

2 TO 

spaces  H (D  and  B'(T,F  (E))  simply  by  and  B'. 

n n p ' ' n n 

One  of  the  key  points  of  our  analysis  is  to  reverse  the  roles  usually 

assigned  to  u and  V;  that  is,  we  shall  consider  u to  be  an  operator 
~ 2 2 

u:  B H which  acts  on  V yieldinq  y according  to  the  relation 

"uCV)  - V(u)  » y.  (4.2) 

Clearly,  such  an  operator’ll  is  linear,  and  it  is  strongly  continuous 
since,  according  to  ''2.44), 


inj(V)ll  T » llV(u)'j  , < exp  (l|u''lp/2p)  |jvH  , 

H‘  "b' 

n n n 


Also,  we  define  the  class  C,  introduced  in  Section 

0 < y1  . 


by 


C - (V  € B^; 


(4.3) 


(4.4) 


B 


where  y a positive  constant  sufficiently  large  for  C to  have  a nonempty 
intersection  with  the  set 

y « (V  € 'u.(V)  ■ V;.  i » I m)  . (4.3) 

Condition  (4.4)  may  be  interpreted  as  a finituOuSS  requirement  on  the  gain 

of  the  system  S (where  we  define  the  system  gain  as  sup  C'IV(u)'|  , /'lu  )V  u) 

L‘(l)  ^ 

and  on  the  smoothness  of  V,  where  the  extent  on  the  relative  boundedness  of 

the  norms  of  the  Frechet  derivatives  of  V up  to  order  n is  determined  by  the 

constants  a.  appearing  in  the  definition  of  HV'  _ (see  Theorem  3,2)  and  by  v. 

2 . " 

Let  p be  a functional  on  B^  with  the  following  properties. 

o 

(i)  p is  bounded  on  bounded  sets  of  B*” ; 


( i i ) p > 0 on  B^ ; 


( lii) 

p(V  + W)  < p(V)  + p(V)  V 

V.  w 

( i v) 

p(  o V)  - |a|  p(V)  Vac 

e.  V 

(v) 

p is  continuous  on  B*“. 

^ n 

We  define  the  error  criterion  e^  over  the  class  C (introduced  in 
Sect  ion  I ) to  be 

e,(V)  - sup  p(\/  - W)  , (4.6) 

W^C 

and  reformulate  Problem  1 as: 

Problem  la.  Same  as  Problem  I,  with  the  additional  specification  that 
Y - H^(I)  and  that  C and  e^  be  as  defined  by  (4.4)  and  (4.6),  '1 

Remark.  Equations  (4.6)  and  (4.4)  constitute  a minmax  criterion  for 
the  choice  of  the  best  V.  This  type  of  criterion  is  particularly  appeal- 
ing when  the  number  of  measurements  is  small  thus  making  a statistical 
criterion  not  plausible.  || 

4.1.  Geometrical  Considerations 

Our  solution  to  Problem  la  relies  on  the  geometry  of  the  set 

.•)  - C ' X . (4.7) 

to  which  we  now  turn  our  attention. 

A set  S in  a normed  linear  space  X is  said  to  be  symmetric  if  there 
exists  an  element  Xq  f S . called  the  center  of  S.  with  the  property  that 
Xq  + ■h  f S •=>  Xq  - " ij  R.  Let  p denote  a seminorm  on  X with  the  properties 

o 

(i)  through  (v)  stated  earlier  (where  now  X replaces  B‘)  then  the  following 
result  holds: 

Lemma  4,1.  If  R is  a bounded  symmetric  set  In  normed  linear  space  X, 

then  the  center  Xq  of  R minimizes 

e(x)  • sup  p(x  - y).  I (4.8) 

yeR 
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Proof.  Since  p is  a function  of  (x-y)  and  S is  symmetric,  we  may 
consider  the  translated  set  S'  ■ S - x^ , which  is  also  symmetric  and  has 
the  center  at  the  origin.  We  shall  prove  that  the  origin  minimizes  e on  ,s‘ . 

Let  [x.l?  , be  a sequence  in  S'  such  that  p(x.)  - e(0)  as  i . 

I I " 1 I 


Then  for  any  e '>  0 there  exists  a N(e)  such  that 
e(0)  < p(x.)  + e.  ' i NCe) 

and 

2p(x.)  - p(2x.)  » p(x.  + y + X.  - y) 

< p(x.  + y)  + p(x.  - y)  - p(y-(-x.))  + p(y  - x.) 
So  we  cone  I ude  that 
either  p(y  - x.)  P(X;). 

or 

If  (4.11a)  is  true,  then 

p(y  - X.)  > p(x.)  > e(0)  - e 
while  if  (4.11b)  holds,  we  have 

p(y  - (-X.))  p(x.)  > e(0)  - c . 

Now  (4,12a)  implies  that 

e(0)  < p(y  - X.)  + e < sup  p(y  - x)  + s 


(4.9) 


p(y  - (-X.))  > p(x.) . 


xfS' 


and  (4.12b)  implies  that 


e(0)  < p(y  - (-X.))  + e < sup  p(y  - x)  + e 

xtS' 


(4,10) 

(4.11a) 
(4. lib) 

(4,12a) 

(4, 12b) 

(4.13a) 

(4. 13b) 


since  X.  and  -x.  are  elements  of  S'. 

I I 

Because  e is  arbitrary,  it  follows  from  (4,13a)  and  (4,13b)  that 
e(0)  < e(y)  for  all  y € S', 

Now  let 

N - (V  t b’  ! tr.(V)  =•  0,  i - I m).  (4,14) 

n * I 


3b 
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The  following  constitutes  a particularization  of  a well-known  minimum  norm 


result  to  our  problem. 


* 2 

Lemma  ^.2,  If  V c B satisfies  the  relations 
" n 


(V.W)2  = 0 VWiCN 


(4.15) 


and 


Tj,(V  ) = y i = I m, 


(4.16) 


then  V is  the  unique  solution  of  the  minimization  problem 


min  j|V'!  - , 


(4.17) 


Vcv  B 


n 

(where  is  defined  by  (4.5)), 
Next,  we  have 


Lemma  4.3.  Let  V denote  the  solution  of  (4,17)  and  T*  be  as  defined 


* 


in  (4.7).  If  V + T1  € fi  then 
★ 


(V  , -n)  2 

B 


(4,18) 


Proof.  Clearly,  V g n . 


'k  "2  2 

To  prove  the  lemma,  suppose  first  that  ]|V  |1  , = y 


Then  since  V is 


a minimizer  on  Q,  it  must  be  that 


liV*  + = V . 

6 

n 


(4, 19) 


wh  ich  imp! i es  that 


(V  , n)  2 = 0 


(4.20) 


B„ 


"tc  2 2 

Suppose  next  that  !lv  l|  . < Y • T^^en  define  the  functional  on 

n 


F(s)  = (V*  + si,  V*  + sm) 

B 

n 


(4.21) 


By  continuity,  there  exists  a neighborhood  [-e,  e]  about  s = 0 such  that 


by  (2.23). 


25 


Proof.  Accordiny  to  the  preceding  lemma,  e , i“l.  ....  m,  are  linearly 

independent  elements  of  l^(E).  The  result  then  follows  from  the  fact 
u.  u.  (u.,u.) 

te  e ' 7^  • " 

P 

It  is  now  possible  to  state: 

Theorem  A. I . In  Problem  la,  suppose  that  the  probing  inputs 
u.;  i-l,  ....  m.  constitute  a set  of  distinct  elements  of  E.  Then  Pro- 
blem la  has  a unique  solution  expressible  in  the  form 


V(*)  - Z c.exp 
i-l  ' 


where  c,  i-  H*  are  determined  by 
i n 


(4.26) 


A c 2 ( t ) 


G y(t)  - G 


/ y.(t) 


(4.27) 


where  C is  the  Gram  matrix  (exp 


(u. ,u.) 


^ i . j = I , ....  m 


set  (exp( ( u . , • ) /? : i 


I . ....  m) 


of  the 


Proof.  Since  u.,  i-1 m,  are  distinct  elements  of  F.,  according  to 

I 

Lemma  4,6  G ' in  (4,21)  exists. 


f.learlv,  by  construction, 

V(u.'i  =y.,  i “ I,  m. 

I I 


If  W c N.  then 


(4.28) 


vV.WN 

B 


r S a.D-*  Z cT(t)  (exp  — ^ 

i j-o  J i-l  ' p 


D-*  W^)dt 


= E P E a,D-‘c^.(t)Djw  (u.)dt  = 0 
i-l  ■'  j-0  J ' ' ' 


(4.29) 


where  the  second  and  third  equalities  follow  respectively  from  Theorem  2.3 
and  the  fact  that  D-'w^(u.)  “0,  j - 0,  i,  ....  n-1,  and  d'^W^(U|)  - 0 a.e, 
since  W(;N.  According  to  Lemmas  4.2  through  4.4,  V(-)  defined  by  (4,2h)  is 

the  center  of  and  hence  by  Lemma  4.1  the  solution  to  Problem  la.  11 
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Remark.  It  is  of  interest  to  obtain  an  estimate  of  the 


' ■ l|V^  - Vj,  11' 


(4.30) 


where  and  correspond  respectively  to  the  actual  system  and  our 
estimate  of  it  according  to  (4.26).  According  to  the  projection  theorem, 
we  have  for  ? 


(4.31) 


If  u.,i  “ 1 m,  are  orthonorma I , then  the  diagonal  elements 

of  G are  e (Napierian  base)  and  its  off-diagonal  elements  equal  to  unity. 
Then,  (4.31)  is  expressible  in  the  form 


7 m 1 ^ .M. 

r.(E)  -»  ">1-  ' , 'T<'>  »j<'> 

P 1-1  I ,J»I  ■’ 


(4.32) 


where 


e + m - 2 , 

Of  “ 2 » S 

e +(m-2)e-(m- 1) 


e + (m-2)e-(m-l) 


(4.33) 


An  estimate  of  » can  then  be  obtained  by  replacing  y foi" 

2 

llVtllp  in  the  above  formula,  where  v is  the  constant  introduced 

in  connection  with  (4.4). 


5 . Causal  Sol ut ion 


Thus  far,  the  only  conditions  assumed  on  the  probing  input-output 
pairs  (Uj,  Vj).  ' ” l,....m,  are  that  they  belong  to  appropriate  spaces, 
and  u.,  i “ l,...,m,  consitute  a set  of  distinct  elements  of  E. 

I 

A 

However,  if  we  demand  that  the  solution  V to  the  system  identification 
problem  of  the  preceding  section  be  "causal",  we  need  to  impose  additional 
restrictions  on  those  pairs  when  m > 1 ; in  other  words,  when  the  number 
of  the  probing  input-output  pairs  is  greater  than  one,  the  causality 


constraint  on  V manifests  as  a set  of  admissibility  restrictions  on  the 
pairs.  These  restrictions  are  developed  in  subsection  5.2.  Also,  in 
that  subsection  we  introduce  a weaker  form  of  the  concept  of  causality, 
which  we  call  "* -causal i ty" ; and  show  that  it  is  possible  to  construct 

A 

an  e-causal  solution  V to  the  problem  under  consideration  under  less 

. ,m, 

I • 

than  for  the  strictly  causal  case. 


stringent  admissibility  requirements  on  the  pairs  (u.,  Yj).  i I 


For  simplicity  in  presentation,  we  will  assume  that  the  opera- 

2 2 
tor  V belongs  to  the  space  B (I,  F (L,(I))  which  we  will  abbreviate  as  B . 

o p z o 

where  I = [0,1],  and  hence  (according  to  (2.46))  admits  a representation 
of  the  form 


y(t)  - V^(u)  - hjj(t)  + JT  J hj(t,Sj)u(S|)dS|  + , ,, 


1 


1 


(5. 


where  ds 


(n) 


ds^  ...  ds^and  the  h.  satisfy  the  conditions  stated 


connection  with  (2.46)  and  (2.47). 
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Let  P^;  U2(l)  - defined  by 


u(s) 


(P^u)(s) 


i f s < t 


if  s > t 


a.e. 


,2 


Def in i t ion  5.1 . V € is  causa  I i f 

Pj(V(P^u))  - P^(V(u))  V t € [0.1]. 

Introduce  the  step  function 
f 0.  t < 0 

w(t) 

ll.  t > 0 . 

<-  2 2 

and  define  the  operator  0:8^  — 8^  by 
(CV)(u)^  - h^(t)  + -jT  J w(t-Sj)hj(t,Sj)u(Sj)dSj 

+ ...  I w(t-s,)...w(t-s^)h^(t,s, 


u(s  )...u(b  )ds 
I n 


(n) 


+ . • # 


(5.2) 


(5.3) 


(5.4) 


(5.5) 


Clearly,  CV  is  a causal  operator.  Furthermore  it  can  be  easily  shown 

that  7 is  a projection  operator.  In  fact,  it  is  obvious  that  C = C,  and  to 

* ^ 2 
show  that  C a C simply  use  the  definition  of  the  inner  product  in 

2 

Denote  by  M the  subspace  of  causal  operators  in  , that  is 


M - f . 

o 

We  seek  the  solution  of; 


(5.6) 


Problem  lb.  Same  as  Problem  la,  except  that  E = Y = L2(l) , and  hence 
2 2 

V € 8 (I,F  (L-(D))  (abbreviated  as  B ) , and  we  require  that  V satisfy  the 
o p 2 o 

uv-  traint 


V € M . 


(5.7) 


^Henceforth,  the  superset  I pt  on  an  operator  symbol  denotes  its  adjoint. 
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Let  y be  as  in  (4.5)  with  B“  replaced  by  . Clearly,  y n M is 

2 

« closed  linear  variety  in  B^.  Hence,  if  y 0 M is  nonempty^ as  we  shall 
assume  from  now  on,  it  follows  from  previous  considerations  that  Problem 
lb  has  a unique  solution  V which  is  the  minimum  norm  element  of  y 0 M. 

A A 

To  obtain  an  explicit  representation  for  V,  we  use  the  fact  that  V 

is  the  unique  element  of  y 0 M orthogonal  to  N n M,  where  N is  the  subspace 

2 ’’ 

defined  by  (4,14)  (with  B replaced  by  B').  This  fact  is  elicited  by 

n o 

Lemma  5.1  below. 

Let  z denote  an  arbitrary  element  of  fl  M,  i.e. 

2 € \ n M.  (5.8) 

C learly , 

X - z + N.  (5.9) 

Lemma  5.1.  ynM»z+(NnM). 

Proof . Suf f Ic  lencv : Let  (using  (5.9)) 

X t Y n M - (z  + N)  n M . (5.10) 

Hence,  according  to  (5.8)  and  (5.9),  x can  be  expressed  as 

X - z + r , (5.11) 

where  z 6 M and  r ^ N , But  since,  according  to  (5.10),  x € M and,  as  we 
have  just  stated,  z € M.  it  follows  from  (5.11)  that  r £ M . So  the  sufficiency 
is  estabi ished. 

Necess  i tv : Let  x be  expressed  as  in  (5.11).  Then,  according  to  the 
right  side  of  Lemma  5.1,  x g M because  both  z and  r belong  to  M,  and  r 
belongs  to  N (because  it  is  in  N (T  M) . Hence,  by  (5.10),  x t ^ ^ M.  !j 
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. I ■ T'lo  Cosp  of  j Si'^ule  Probirq  Input -Output  Pn  i r 


For  the  case  of  a s i mj  I e piobini;  Inpjt- juiout  pair,  .ic  havi' 
1 heoi'>-7:  3.1.  If  m !,  the  sc  hit  i on  tc  Piohlen  1 is  of  the  loir 


t 


V I'u)  =■  C,('-)  '<P'  ~ 11  , ( sluf  S/OS''  , 


f ' . r ) 


Khei  o 


■,(t/  ^ (expf-  - ^ lu|(s)|  os'll  II 


(5.  ! i) 


Proof.  It  is  clear  that  (5.13)  is  a feasible  soiotion.  Accorciiec; 
to  Le'.inu  5.1,  to  show  that  it  is  of  m in  iinuni  norri  it  Is  sufficient  to 
prove  that  Such  V is  orthoconal  t i Nt'M. 

(5.13)  can  ..e  expanded  in  the  form 


- C (t)  (1  + — „ w(t-s  )u  (s  lut's,  .'us.  r 
- ' • 0 II.  ■ I 


I 1 


+ — ~ I 


3 ; 0 ■ 0 0 


w(  t - s I ) w(  t - s.,)u  I ( S , 1 U . f s S I i u(  S , 'ds  ^ tls,^ 


(5,  14) 


• il-,  hng  the  inner  product  in  8^  of  thu  aho.e  V with  an  arbifiary  V 


Ob  tain 


rcoieserted  by  (5.5)  oelonyino  to 
yV,  ■//:  ; :-|(t)  >h^(t)  h yr  J ■i,(r,jj)w(t-i,  ) .i.(s,)ds 


i r 


I 


1 i 


+ — '*  r b,/t.s,,s  ) w(i-s,)  w(t-s,) 
- • 0 ■ 0 ‘ ^ 


U|(S|)U|  (s^)  dS|ds,,  + . . .'idt 


(5.15) 


i" 

H 
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In  (S.15),  btCJuse  VfM,  sve  have 

h|( t .s  |)  w( t-S| ) = h| ( t .s I ) a a.  (j.ina) 

w(t-S|)  w(i-«;^)  » h^  (t.s^.s,)  a e.  (5. lob) 

etc . , . 

Using  equations  (5.16).  expression  (5.15)  reduces  to 
1 

^ c (t)  V (u.)  dt  . (5.17) 

0 

Above.  \/^(U|)  “ 0 a.e.  because  7--.N.  Hence  (5.17)  vanishes  thus 
establishing  that  V defined  by  (5.12)  is  orthoconal  to  NfM  . jl 

5.2  The  Case  of  Several  Probing  Inout-Oiitput  Pairs 

We  now  consider  the  case  in  which  the  number  of  the  orobiiui 

input-output  pairs  (u.,Y.),  i l....,m,  is  equal  to  or  greater  than  tv/o. 

We  denote  col(u, u ) and  col(y, y ) respectively  by  “u  and  "v  , 

I m I Ml 

and  introduce  the  causal  Gram  matrix  ti(t)  with  elements  defined  by 
~ I ^ 

G..(t)  - expl  - r u.(t')  u,(t’)  dt  ) i,i=>l m.  (5.1'' 

'J  3 ■ 0 I J 

5,2.1  Strictly  Causal  Solution 

As  in  the  case  of  (4.27)  (with  G now  replaced  by  the  causal 


G(t)).  we  require  to  obtain  the  solution  c(t)  = col(c‘,(t) (t))  of 

I ni 

the  equation 

'(jt  t ) c(  t ) = 'yt ) ) .a.e.  (5.19'i 
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The  difficulty  arises  at  the  origin  because  TTCO)  is  a singular 
matrix  with  all  elements  equal  to  unity.  According  to  this  last  fact, 

in  order  for  (5.19)  to  have  a solution  at  t ■ 0.  y.(0),  i ■ I m, 

n 

must  be  well  defined,  and  there  must  hold 


c,(0)  + c„(0)  +•••+?  (0)  - y,(0)  - y,(0) 
I m I J 


y (0)  . (5.20) 


In  addition,  the  following  condition  will  be  needed  in  the  neighborhood 
of  the  origin; 

For  some  t > 0 , the  rest  rictions  of  u.  to  (0,t),  0<  t <t,  are 

2 

distinct  elements  of  L (0,t)  and  ' 

I im  G Vt)  y(t)  exists  as  a finite  vector(  where, in 
t-0 

tX)  taking  the  limit, we  consider  an  appropriate  member  of  the 

equivalence  class  y). 

We  will  call  (5.20)  and  (5.21)  a set  of  "admissibility  conditions"  on 
the  probing  input-output  pairs. 

Remark.  For  m * 2,  condition  (5.21)  is  obtained  by  requiring 


(5.21) 


y,(0).  Y,(0).  u (0).  and  u^(0)  be  well-defined 


and  finite  and  Uj(0)  ^ U2(0). 


(5.22) 


This  is  gleaned  from  the  fact  that  (for  m - 2> 


G (t)  y (t)  - G (t)  y,(t) 

T ^ 

G,,(t)  G22(t)  - G,2(t) 


(5.23) 


G^^(t)  y^(t)  - G^^(t)  y^(t) 

G,i(t)  G2,(t)  - 5j^(t) 


(5.2.'4'> 


* If  y.cL^(P,  theny.  's  an  equivalence  class.  For  the  sake  of  complete 
general i ty . we  say  that  yl(0)  Is  "well  defined"  if  a member  y.  of  the  class  y. 
is  such  that  y.(t)  tends  io  a limit  as  t « 0.  and  we  denote  this  Hmlt  by  y (0l, 
In  other  words!  y.  is  the  equivalence  class  generated  by  such  a y..  A similar 
statement  applies'when  we  say  that  a derivative  of  yj  is  well  defined  at  0. 
c(0)  Is  defined  as  the  solution  of  (5.19)  with  y(Q)  defined  as  above. 
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Now,  according  to  (5.I8>, 


tim  G. .(t>  - 1,  1,  j - 1 m (5.25) 

U 

t - 0 

I Itn  g!  .(t)  - 1 u,(0)  u.(0)  (5,26) 

t - 0 

Hence,  by  I'Hopital's  rule, 

G22(t)  y (t)  + G22(t)  y'(t)  - G,'2y2^‘’  ’ ^12^^^ 

lim  c (t)  • 11m  ^ ^ ^ 

t - o'  t-0  Gj,(t)  G22(t)  + G||(t)  G22(t)  - 2G,2(t)  Gj2(t) 

r U2(0)  (U2(0)  - U|(0))  y|(0)  + y|(0)  - yj(0) 

« “ ■ ■ ’ I - - - — — p — . • . , . ..I  .1  _i  — . _i 

' (u^(0)  - U2(0))^  (5.27) 

and  similarly  with  C2(t),  which  shows  that  (5.22)  is  a sufficient 
condition  for  (5.21). 

For  the  case  of  m " 3,  the  above  type  of  calculation  becomes 
extremely  tedious.  In  application  of  I'Hopital's  rule,  different 
at  ion  up  to  order  three  has  to  be  carried  out  of  the  rtunjerator 
and  denominator  of  the  expressions  for  c.(t),  i ■ 1,2,3.  Such  a 
differentiation  of  the  denominator  gives 


' > ' 

0 (t) 


dt‘ 


and  hence 


d"'(0)  . \ 


;ii(t) 

Gl2(t) 

G,3(t) 

;2i(t) 

G22(t) 

G23(t) 

• 

;3i(t) 

S32(t^ 

G33(t) 

1 

' 1 

l^^l  c 

h' 

“3’ 


<"l"2  ^ "I"2  ^ P "2  ' 

, , ' ' 12  2 
! (UjU,  + u,u^  + - U,U3  U3  I 


(5.28) 


(t) 


I I 12  2 

U]  (u,U2  + UjUj  + p u,u,) 


+ (Uj  - u^)  (2  + - u,) 

1 2 

“3  ^“2^3  “2^3  '*’  P ^2^^ 


. / ' ' 122, 
u,  1 (u,U3  + u,U3  + j u,U3) 

+ (u,  - u^)  ^2  ' ^"2“i  ^ "^3  ^ 

U3  I (2U3U3  + } U3) 


(5.29) 


where  we  have  abbreviated  u.(0)  and  u’(0>  by  u and  u'  for  i « 1,2.3. 

I I II 

From  (5.29)  it  follows  that  d'"(0)  ^ 0,  if  Uj(0)  ^ u^fO)  ft  03(0). 
This  and  considerations  resulting  from  differentiating  the  numerator 
lead  to  the  ‘’allowing  sufficient  condition  for  (5.21)  to  hold:* 


y((0),  u.(0),  and  u.'(O),  i - 1,2,3,  be 


well  defined  and  finite  and  u.  (0)  Uj(0)v  l)fj  , i,j  • 1,2,3. 


(5.30) 


A sufficient  condition  similar  to  (5.30)  can  be  derived  far 

m '*  3 but  the  calculation  is  too  tedious  to  be  carried  out  here.  " 

, 2 

Lemma  5 .■•  . Suppose  u.  and  y.  “L  ( I)  , i • 1 m,  satisfy 

conditions  (5.20  and  (5.21'.  Then  (5.19)  has  a unigue  solution  ‘c  with 

c.-  L^(I).  ” 


Proof.  Since  lim  G (t)  y(t)  exists  and  is  finite,  there  is  a 
t-0 
t''0 


sub  interval  {0.1^1,  0 <l|  <t,  over  which'c(t>  • G (l)  v(t)  is  bounded 
almost  everywhere.  Since  u.  , 1“  I,  . . . . m,  are  distinct  on  (O.Aj,  thev 


* In  this  ca  Icu  1 at  Jn.  h I ghe  r order  dor  i v .it  I ves  (in  ordin.irv  or 
distributional  sense'  of  u.  and  y.  can,.»l  out  at  0. 
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are  distinct  on  [O.t],  t^<  t < I . and  hence  G and  G’'are  continuous 

2 

on  This  and  the  fact  that  y|^  L (I)  establishes  that  the  restriction 

^ 2 

of  C|to  [tpl]  belongs  to  L ( tp  I) . Because,  in  addition.  ? is  bounded 
a.e.  on  [0,t^].  we  co'dude  that  c- ^ L‘’(I).  '' 

The  above  leads  to  the  following: 

Theorem  5.2.  Let  m > 3 in  Problem  lb.  If,  in  this  problem,  we 
further  restrict  the  operator  to  be  identified  V to  be  such  that 
(u.,y.),  i » l,...,m,  satisfy  the  admissibility  conditions  (5.20)  and 
(5.21),  then  Problem  lb  has  a unique  solution  expressible  in  the  form 

A A 

(with  the  understanding  of  the  notation  V(u)(t)  = V^(u)): 

. m (u. , ,) 

VjC)  - r c.(t)  exp[  — 5 I , (5.31) 

i ” I 

where 

- (•,•)  , , (5,32) 

L-(O.t) 

and  c'.  are  the  components  of  the  vector  ? obtained  by  solving  (5.19). 

- 2 

Proof.  It  is  clear  that  V € B and 
o 

(\/u.)(t)  = y.(t),  i » l,,.,,m  . (5.33) 

Proceeding  in  much  the  same  way  as  in  the  proof  of  Theorem  5.1, 

we  can  show  that  each  term  of  the  form  c.exp[  — g 1 is  orthogonal 

to  every  V 6 N 0 M,  this  completing  the  proof.  '| 


! 


5.2.2.  c-Causal  Solution 

In  order  to  weaken  the  admissibility  conditions  (5.20)  and 
(5.21),  we  introduce  the  following  generalization  of  Definition  5.1: 

2 

Definition  5.2.  is  £-causal  (for  some  positive  e < I)  if 


(V(P^u))  = Pj.  (V(u))  V t > e l|  . 


(5,34) 


We  may  now  formulate  the  following  weaker  form  of  Problem  lb; 


Problem  Ic.  Same  as  Problem  lb  except  that  the  constraint  (5.7) 
is  replaced  by  the  restriction  that,  for  some  specified  j,  0 < g < I, 
V be  non-causal  for  0 <;  t s g and  causal  for  t > g . 1| 

Theorem  5.3.  Let  g , 0 < g < 1 , be  such  that  the  restrictions  of 

2 

Uj,  i = l,,..,m,  are  distinct  elements  of  L (0,g).  Then  for  this  g. 
Problem  Ic  has  a unique  solution  described  by 


V (•)  =?  c (t)  V 0 < t < g 


(5.35a) 


m r 

V,(-)  = I c.(t)  exp(— ^ '')  V < t < 1, 

t.l 


(5.35b) 


c.(t)  =•  G ' 7(t), 


(5.36) 


c.(t)  = r'(t)  y(t)  , 


(5.37) 


where  G and  C'(t)  are  mxm  matricsswith  elements 
G|j  - expl, p-i ) 


(5.38) 


1 


1 


j 
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6.  Nonlinear  System  Identification  in  the  Noisy  Case 


We  next  turn  our  attention  to  Problem  2 formulating  it  in  the  following 
specific  form: 

Problem  2a.  Find  the  solution  V of 


min  '^(V)  , 
V € 


(6.1a) 


where 


^(V)  = liVjl^^  + 2 q:'!lu.(V)  - y.ll^Y 

B i = l 
n 


(6. lb) 


Remark:  The  class  properties  of  V are  taken  into  account  by  the  con- 
stants a.,  i=l,  ...,n,  (see  equation  (3.6))  picked  in  the  definition  of  the 
2 

inner  product  in  , while  the  noise  properties  are  accounted  for  by  the 

weights  q.',  i = l,  ....  m,  in  (6.1b).  ]| 

We  seek  a noncausal  (not  necessarily  causal)  solution  V of  (6.1). 

Details  on  the  causal  extension  of  the  present  results  can  be  worked  out  in 

exactly  the  same  way  as  in  section  5 and  hence  will  be  omitted. 

To  obtain  the  solution  to  Problem  2a,  we  first  construct  a Hilbert  space 

B defined  by  the  Cartesian  product 
m tjmes 

B = B^  X Y X ...  X Y>  (6.2) 

n 

with  the  inner  product  between  any  two  elements  of  B f=col(f„,f, f ) 

U I m 

2 

and  g=col(gQ ,gj , . . . ,g^) , where  fQ  and  g^  $ and  f.  and  g.  g Y,  i=l m, 

defined  by 

.1 

(^,g)^  = (fo-gg)  2 “’i  ^^i’SiV  ' 

B i = 1 
n 

2 ^ 

Introduce  the  operator  L;  B in  the  form 

L = col(l,u,(-) (6.^) 

and  denote  by  y the  yector 


y = col(0,yj yj. 


(6.5) 


Now  the  set 
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7 . Computer  Simulation  Example 

In  order  to  illustrate  the  preceding  developments,  we  consider  the 
simple  example  of  a system  described  by  the  scalar  differential  equation; 


y(t)  • -y(t)  + u(t)y(t),*“  d/dt,  y(0)  - 1,  (7,1) 

on  the  interval  I - [0,1],  The  exact  input-output  description  for  such  a 
system  is 


y(t) 


t 

V (u)  » exp  (T  (u(s)  - l)ds} 

^ t) 

t 

exp(-t)  exp( f u(s)ds} . 

‘ 0 


(7.2) 


We  assume  we  are  given  the  eleven  probing  input-output  pairs: 


u,(t)  - 1 
U2(t)  = sin  2TTt 
U2( t)  - sin  4rrt 


y,(t)  - 1 

y2(t)  = exp(-t)expC-(cos  2nt-l)/2Tt} 
y2(t)  “ exp(-t)exp(-(cos4TTt-l)/2TT} 


u, ( t)  - sin lOnt 
o 

U2(t)  " cos2TTt 
Ug(t)  “ cos4nt 


y (t)  » exp(-t)exp(-(cosl0nt-l)/2n} 
o 

= exp(-t)exp((sin2TTt)/2TT} 
yg(t)  •=  exp(-t)expC(sin4rrt)/2rrl 


Ujj(t)  » coslOnt  ^11^*’^  “ exp(-t)exp((sinl0nt)/2Tr} 

(7.3) 

With  p - 1 and  u.  . , i - 1,  .,,,  11  defined  as  above,  (5.18)  becomes 
t 

lj..(t)  - exp(r  u.(s)u.(s)ds}  , 

•J  "o  ' J 

' 1 j”' t • . . . II  • (7.4) 

It  Is  clear  that  the  (U|,y.),  i = l,...,m,  satisfy  the  condition  for 

A 

an  s-causal  solution  V to  exist,  given  by  the  expression  (5.35),  for 
arbitrarily  small  e (see  Remark  after  Theorem  5,3). 

Consider  the  test  input  u(t)  ” t.  According  to  (7.2),  the  corresponding 
exact  output  is 


J 
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y(t)  » exp(-t)  exp(^  t ) 


(7.5) 


Table  I and  Fig.  1 compare  samples  of  this  output  at  sampling  instants 


t . « (.04)  i , i = 1 25  , 


(7.6) 


with  the  corresponding  samples  of  the  output  of  our  solution  operator  V 


in  a digital  computer  simulation,  where  we  have  assumed  the  value  of 
e » .04.  The  agreement  is  remarkably  good  except  for  the  first  two  instants. 
The  disagreement  near  t = 0 results  from  the  fact  that  det(G.j(t))  tends  to 


zero  as  t 


If  we  take  e = .12,  the  agreement  is  nearly  perfect. 


TABLE  I 


t. 

y(t.) 

y(t.) 

' y(t,) 

.04 

.96155838 

2.80182753 

-191 

.08 

.92607505 

.50049397 

45.95 

.12 

.89332931 

.89325025 

0.00885 

.16 

.86312134 

.86312003 

1. 51x10“^ 

.2 

.83527021 

.83527024 

-3.59X10'^ 

.24 

.80961213 

.80961212 

1.23X10'^ 

.28 

.78599881 

.78599882 

-1.27x10"^ 

.32 

.7642961 

.7642961 

0 

.36 

.7443827 

.7443827 

0 

.40 

.72614904 

.72614904 

0 

.44 

.70949630 

.709496 30 

0 

.48 

.69433550 

.69433550 

0 

.52 

.68058674 

.68058674 

0 

.56 

.66817845 

.66817845 

0 

.60 

.65704682 

.65704682 

0 

.64 

.64713523 

.64713523 

0 

.68 

.63839376 

.63839376 

0 

.72 

.63077882 

.63077882 

0 

.76 

.62425272 

.62425272 

0 

.80 

.61878339 

.61878339 

0 

.84 

.61434415 

.61434415 

0 

.88 

.61091344 

.61091344 

0 

.92 

.60847467 

.60847467 

0 

.96 

.60701608 

.60701608 

0 

1.00 

.60653066 

.60653066 

0 

8 . Cone  I us  ion 

In  this  paper  we  have  developed  an  approach,  based  on  a Fock  space 
and  Volterra  expansion  framework,  for  the  formulation  and  solution  of 
the  nonlinear  system  identification  problem,  both  without  and  under  the 
causality  and  g-causality  constraints. 
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